MCAR: The Happy Case

• Missingness is truly random.
• For MCAR, we randomly delete nutrition X for a subset of mothers.
• Random missingness has no relationship to education or nutrition values.

MCAR: Impact

• Missing data does not cause biased estimates. It just reduces precision/power.
• Example: random equipment failures, accidentally dropped samples.

MCAR: Dag

Simulation Code: Base Data and MCAR

# Ensure base data exists (5000 rows defined earlier)
if (!exists("sim_data")) {
  sim_data <- simulate_birth_data(
    n         = 5000,
    beta_x    = beta_x_true,
    beta_e    = beta_e_true,
    sd_error  = sd_error_true
  )
}

# Full data model (gold standard) on the complete data
full_fit <- lm(Y ~ X + E, data = sim_data)

# Randomly mark 50% of X as missing (MCAR) for this demo
mcar_data <- sim_data |>
  mutate(
    drop_flag = rbinom(n(), 1, 0.5),
    X_mcar = if_else(drop_flag == 1, NA_real_, X)
  )

MCAR with Listwise Deletion

# Only keep rows where X_mcar is observed (listwise deletion)
mcar_cc <- mcar_data |> filter(!is.na(X_mcar))
mcar_fit <- lm(Y ~ X_mcar + E, data = mcar_cc)

nrow(mcar_cc)

[1] 2522

summary(mcar_fit)

Call:
lm(formula = Y ~ X_mcar + E, data = mcar_cc)

Residuals:
    Min      1Q  Median      3Q     Max 
-7.7022 -1.3534  0.0086  1.3901  7.7041 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  1.97448    0.05698   34.65   <2e-16 ***
X_mcar       0.47749    0.04102   11.64   <2e-16 ***
E            1.02528    0.10239   10.01   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.043 on 2519 degrees of freedom
Multiple R-squared:  0.1916,    Adjusted R-squared:  0.1909 
F-statistic: 298.5 on 2 and 2519 DF,  p-value: < 2.2e-16

Summary of Differences

Call:
lm(formula = Y ~ X_mcar + E, data = mcar_cc)

Coefficients:
(Intercept)       X_mcar            E  
     1.9745       0.4775       1.0253  

# A tibble: 2 × 3
  method            estimate     se
  <chr>                <dbl>  <dbl>
1 Full data (truth)    0.500 0.0288
2 MCAR listwise        0.477 0.0410

Power Analysis: MCAR vs. Complete Data

MCAR: estimates center on truth; uncertainty widens as more X is missing.

MCAR: Not a Big Deal But Rarely Plausible

• MCAR is the happy case: just drop mothers with missing nutrition, move on.
• You lose precision/power, but no bias.
• Problem: MCAR is seldom plausible, except in a few cases (e.g., random measurement instrument failure)

MAR: Nonresponse Driven by E

• Missingness rises with an observed factor (e.g., education, clinic context)
• Education affects both nutrition and birthweight.
• Dropping cases with missing values up-weights higher-education (for example) mothers, biasing the nutrition effect.

MAR Scenario Set

We will look at:

• A single “truth” dataset (n = 20,000) with no missingness
• MAR cases with 10%, 30%, and 60% of X missing
• Missingness depends on E (observed)
• All summaries and diagnostics in this MAR section restrict to those E-driven scenarios
• Next section: reuse the same data but let missingness depend on X to show why that mechanism becomes MNAR

MAR DAG

MAR Scenarios, summarized

• No Y/E bias here; we only are missing on X here
• Bias increases with more missing data
• Bias increased when relationship between E and p(missing) is stronger
• The highest bias scenario has a particularly strong link between the true value of X and p(x missing)

MAR Scenarios: Naive Regression (Y ~ X)

• No adjustment for E; drop rows with missing X.
• True \(β_X\) = 0.50
• E affects both Y, X, and p(x missing); these models are mis-specified without E
• MNAR and mis-specified models tend to go hand-in-hand – you are missing key variables in your overall causal structure

MAR Scenarios: Adjusted Regression (Y ~ X + E)

• This model is correctly specified. “No backdoor paths.”
• Observed scenarios drop cases with missing X.
• True \(β_X\) = 0.50
• Even without any imputation, the adjusted model recovers \(β_X\) across scenarios.
• The main consequence is power loss, not bias.

When MAR Holds (and When It Doesn’t)

• If we model the variables that drive missingness (here, E), listwise deletion analyses stay near the truth.
• In that case, missing data mostly hurts precision: fewer cases translates to bigger SEs.
• Trouble begins when missingness of a variable depends on its own value: then MAR is violated relative to the analysis, and bias appears.
• In real workflows we can combine modeling plus imputation; this performs best.

Common, Non-Ideal Imputation Tactics

• We apply four historically common tactics to each MAR scenario (E-driven at 10/30/60% missing) plus the full, no-missing baseline.
• Every regression keeps \(E\) in the model (since we observe it); what changes is how we reconstruct \(X\).
• Tables on the next slides show \(β_X\), standard errors, and how far each method drifts from the true 0.50.

Mean Imputation

• Insert a single cohort-wide mean for every missing nutrition value.
• Mechanics: compute the mean of observed \(X\), then plug that constant into each missing slot
• All imputed mothers receive the same nutrition score, so their points line up on the same vertical slice of the scatter.

Mean Imputation: Simulation Results

Mean Imputation vs. Listwise Deletion

Mean Imputation: Problems

• Even with \(E\) in the regression, bias is worse with mean imputation than just using listwise deletion.
• In this case, shrinks the slope toward zero: missing cases on average have worse nutrition; we replace those missing values with the data from cases where highly educated, well-nourished mothers are over-represented.
• Standard errors are too small because we flooded our data with identical, non-varying values.

Mean + Indicator

• Start with mean imputation, then append a binary “missing” indicator to the regression.
• Mechanics: fill missing \(X\) with the cohort mean, create miss_ind = 1 when \(X\) was imputed, and fit \(Y \sim X + E + miss\_ind\).
• Intuition: mothers with missing nutrition get the same slope as everyone else but a separate intercept shift governed by the indicator.

Mean + Indicator: Simulation Results

Mean + Indicator vs. Listwise Deletion

Mean + Indicator: Problems

• In this case, adding an indicator reduced bias somewhat compared to mean imputation, especially when missingness was lower, but bias remains
• Treats missingness as a fixed effect, so can exacerbate bias on other coefficients (e.g., \(\beta_E\)) correlated with missingness

Hot-Deck Imputation: Mechanics

• For each missing nutrition value, randomly draw a donor mother who reported \(X\) and copy her score.
• Mechanics: sample with replacement from observed \(X\)’s, replace the missing Xs with thoes values, and analyze \(Y \sim X + E\) on the filled in dataset.

Hot-Deck Imputation: Simulation Results

Hot-Deck Imputation vs. Listwise Deletion

Hot-Deck Imputation: Problems

• In this case, performs worse than mean imputation, vastly exacerbating bias from listwise deletion
• Works when missingness is MCAR; once it depends on \(E\), donor pools misrepresent the missing cases.
• Still leads to overly tight SEs because draws are constrained to known cases. In our case, these known cases are from a tighter distribution than the truth.

Regression Imputation: Mechanics

• Estimate a predictive model \(\widehat{X} = \widehat{\alpha} + \widehat{\gamma} E\) using only mothers with observed nutrition.
• \(\widehat{\alpha}\) is the fitted intercept (baseline nutrition when \(E = 0\)); \(\widehat{\gamma}\) is the estimated effect of \(E\) on nutrition.
• Mechanics: use the observed \(E\) to predict \(X\) for every missing case, replace the missing Xs with that prediction, and then run the outcome regression.
• In our simulation, low-education mothers receive imputations near \(\widehat{\alpha}\) while high-education mothers get \(\widehat{\alpha} + \widehat{\gamma}\)—mirroring the DAG structure instead of collapsing to a single mean.

Regression Imputation: Mechanics (Multiple Missingness Causes)

• When several observed variables \(Z_1, Z_2, \ldots, Z_p\) relate to nutrition or missingness, fit \(\widehat{X} = \widehat{\alpha} + \sum_{j=1}^p \widehat{\gamma}_j Z_j\) among mothers who reported \(X\).
• Each coefficient \(\widehat{\gamma}_j\) shows how that covariate shifts expected nutrition; the imputed value is the linear predictor for each missing case.

Regression Imputation: Simulation Results

Regression Imputation vs. Listwise Deletion

Regression Imputation: Problems

• SEs are typically too optimistic: every missing X is deterministically set
• Multiple imputation could add back the right amount of noise and uncertainty.
• Remember: the goal is unbiased estimates of both \(\beta_X\) and \(SE(\beta_X)\)

Deterministic Imputation ≠ Power

• Same line, more dots: Regression imputation deterministically replaces each missing \(X\) with its predicted value from \(E\). Those rows reinforce the complete-case relationship instead of adding new leverage.
• False confidence: You can see how SEs mirror the omniscient model, but they should in fact be larger than the omniscient model.

Where Do We Go From Here?

• We need to put believable spread back into the missing rows rather than printing the same predicted value over and over.
• Multiple imputation helps mitigate this problem

Multiple Imputation: Mechanics

• Treat the missing nutrition scores as random draws from a model that predicts \(X\) using the information we do have (here, \(E\)).
• Do that filling process multiple times, fit \(Y \sim X + E\) in each completed data set, and then average the estimates using Rubin’s rules.

Multiple Imputation: Simulation Results

Multiple Imputation vs. Listwise Deletion

Why Multiple Imputation Helps

• Adds plausible wiggle: MI draws \(X^* = f(E) + \varepsilon\), where \(f(E)\) is the best guess for nutrition given \(E\) and \(\varepsilon\) is a random residual sampled from the estimated spread. That restores the variation we would have seen if \(X\) were observed.
• Honest uncertainty: Rubin’s imputation combination rules blend the within- and between-imputation variance, so SEs stay truthful even when missingness is heavy.
• Bias correction: Because each draw reflects how \(X\) and \(E\) relate, the averaged estimate lands near the true \(\beta_X\).

Common Mistake: Not Imputing Outcomes

• When missingness depends on Y, analysts often hesitate to use Y to impute X
• But if you leave Y out you replicate the listwise bias because the imputations can’t see the selection mechanism.

“Isn’t \(Y\) Later in Time?”

• Yes. Nutrition is recorded before birthweight, but by the time we analyze the data, \(Y\) is observed for every mother.
• Imputation models the joint distribution of \((X, Y, E)\) using all observed information; it doesn’t send signals back in time.
• Leaving \(Y\) out because of timing hides the mechanism that made \(X\) go missing and recreates the listwise bias.
• The causal analysis still honors temporal order: first repair the dataset using all available information, then fit \(Y \sim X + E\) on the completed data.

Outcome-Dependent Missingness: New Scenario

• In this new scenario, nutrition still causes birthweight, but now recording nutrition depends on the outcome itself: emergency low-birthweight deliveries lose the time needed to capture \(X\).
• Missingness is MAR because it relies on observed \(Y\) and \(E\), yet complete-case regression drops precisely the low-\(Y\) mothers and tilts \(\hat\beta_X\).
• Multiple imputation repairs the bias by letting \(Y\) (and \(E\)) predict the missing \(X\) values before fitting the usual \(Y \sim X + E\) model.

DAG: \(Y\) Drives Whether \(X\) Is Observed

Simulation Summary

• \(n = 6{,}000\) mothers; true \(\beta_X = 0.50\)
• About NA of nutrition scores are missing when birthweight is low
• Listwise deletion keeps only 3345 observations and overstates the nutrition slope

Listwise vs. MI

• Listwise deletion: drops low-\(Y\) deliveries, depressing the slope of X
• Multiple imputation: models \(X\) with predictors \((Y,E)\), fills 15 completed datasets with predictive mean matching, and Rubin’s rules recover the true slope and honest SEs.
• The outcome still depends on the same covariates (\(X\) and \(E\)); MI just leverages outcome information during imputation instead of discarding it via listwise deletion.

Multiple Imputation: Takeaways

• MI lands on the true \(\beta_X\) even when missingness depends on E, provided we include the variables (here, \(E\)) that explain that missingness.
• The combined SEs stay honest because Rubin’s rules keep the run-to-run variation in the mix.
• Next session we’ll dig into more details on how MI works and how to implement it yourself.

MNAR-lite Setup

• Same 20,000-mother dataset as before
• Missingness now rises when nutrition is low even after we condition on E; missingness depends on X.
• Analysts observe \(Y\), \(E\), and an \(X\) that is sometimes missing. Because the chance of missingness rises when the true \(X\) is low, the mechanism is technically MNAR.

Why OLS Survives This Scenario

• The missingness in the DAG depends only on the regressors (\(X\) and \(E\)), not on any unmodeled part of \(Y\).
• That means the complete cases still satisfy the correct outcome relationship: \(E[Y \mid X, E] = \beta_X X + \beta_E E\).
• Translation: even with MNAR missingness on \(X\), listwise deletion remains unbiased as long as the outcome model includes the variables driving the missingness. You lose precision, not correctness.

MNAR-lite: Selection Snapshot

• Education means stay similar, but the observed nutrition mean inflates as missingness rises
• These differences confirm the selection is MNAR relative to nutrition, even though our regression can still recover the truth with the right control set

MNAR-lite: Naive Regression (Y ~ X)

• Dropping rows with missing \(X\) and ignoring \(E\) yields the expected MNAR bias
• Higher missingness exaggerates the up-weighting of well-nourished mothers, so slopes drift up

MNAR-lite: Adjusted Regression (Y ~ X + E)

• Conditioning on \(E\) blocks the backdoor path, so even listwise deletion stays near the true \(\beta_X = 0.50\)
• The only cost is power loss; SEs grow as missingness increases

MNAR-lite: Listwise vs. MI

• MI lines up with listwise deletion because both models respect the same ignorability structure
• Rubin’s rules still help by keeping

MNAR & Imputation

• Multiple imputation is not a magic bullet
• If you do not have rich enough covariates to accurately model p(missing X), there is no missing data strategy that can save your analysis.
• Let’s finish by examining such a scenario

MNAR DAG: Hidden Drivers

MNAR Stress Test Setup

• Nutrition, income, provider quality, family stability, and class are all correlated.
• We only observe education and insurance, so any missingness tied to the other pieces is effectively MNAR for us.
• Let’s compare listwise deletion vs. MI when we have a complex causal graph with unobserved data

MNAR Simulation Details

• Simulated 2,000 mothers with seven covariates; truth: \(\beta_X = 0.50\) in \(Y \sim X + \text{education} + \text{insurance}\).
• Missing nutrition is more likely for low income, low provider quality, fragile family, lower class position mothers.
• Analysts only get \(Y\), education, insurance, and nutrition (with missingness).

MNAR Table 1: Who’s in Each Sample?

• The MNAR-lite illustration showed how listwise deletion recovered accurate model estimates, but our Table 1 is still wrong
• We can see here by using MI, our summary statistics are more accurate than listwise deletion, but still biased.

MNAR Model Specs

• Truth (omniscient): Y ~ X + education + insurance + income + provider + family + class.
• What we can actually fit: \(Y \sim X + \text{education} + \text{insurance}\) on whatever \(X\) values we have.
• We compare three versions of that observed model: full data (for reference), listwise deletion, and MI.

MNAR Results: Listwise vs. MI

• Omniscient model (with hidden drivers) nails \(\beta_X\), as expected.
• Even with full nutrition but no hidden drivers, we drift because the omitted variables matter.
• Once nutrition goes missing, both listwise and MI lean on education/insurance; MI can actually look worse than listwise deletion!

MNAR: No Easy Fix

• Both listwise deletion and MI miss the truth because neither sees the hidden drivers of missingness.
• MI can even overshoot when missingness leans on factors we never measure—its imputed \(X\) just mirrors education/insurance again.
• Only extra information (new variables, external data, or sensitivity analyses) can break out of this corner.

Takeaways

• Handling missing data well is all about good causal inference, on both Y and your missing Xs
• No missing data strategy can fix an impoverished data set
• You need to spend a good deal of time thinking about your causal graphs/DAGs before you implement imputation strategies
• Don’t be afraid of simulating your specific data; you can’t observe your missing Xs, but you can simulate scenarios that tell you what would happen to your results under various conditions.

Thank You!

• Erik Westlund
• Johns Hopkins Biostatistics Center
• ewestlund@jhu.edu